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ABSTRACT 

Background The frequency of outpatient 
diagnostic errors is challenging to determine due 
to varying error definitions and the need to 
review data across multiple providers and care 
settings over time. We estimated the frequency 
of diagnostic errors in the US adult population by 
synthesising data from three previous studies of 
clinic-based populations that used conceptually 
similar definitions of diagnostic error. 
Methods Data sources included two previous 
studies that used electronic triggers, or 
algorithms, to detect unusual patterns of return 
visits after an initial primary care visit or lack of 
follow-up of abnormal clinical findings related to 
colorectal cancer, both suggestive of diagnostic 
errors. A third study examined consecutive cases 
of lung cancer. In all three studies, diagnostic 
errors were confirmed through chart review and 
defined as missed opportunities to make a timely 
or correct diagnosis based on available evidence. 
We extrapolated the frequency of diagnostic 
error obtained from our studies to the US adult 
population, using the primary care study to 
estimate rates of diagnostic error for acute 
conditions (and exacerbations of existing 
conditions) and the two cancer studies to 
conservatively estimate rates of missed diagnosis 
of colorectal and lung cancer (as proxies for 
other serious chronic conditions). 
Results Combining estimates from the three 
studies yielded a rate of outpatient diagnostic 
errors of 5.08%, or approximately 12 million US 
adults every year. Based upon previous work, we 
estimate that about half of these errors could 
potentially be harmful. 

Conclusions Our population-based estimate 
suggests that diagnostic errors affect at least 1 in 
20 US adults. This foundational evidence should 
encourage policymakers, healthcare 
organisations and researchers to start measuring 
and reducing diagnostic errors. 



BACKGROUND 

Diagnostic errors pose an important 
threat to healthcare quality and safety in 
outpatient settings. 1 2 The frequency of 
outpatient diagnostic errors is unknown, 3 
due in part to varying definitions across 
studies and to the challenge of detecting 
these errors, which typically emerge 
across multiple episodes of care over 
time. A recent review by Graber dis- 
cussed the use of several methods to 
study diagnostic errors, including autop- 
sies, case reviews, surveys, incident 
reporting, standardised patients, second 
reviews and malpractice claims. 3 
However, most of these methods have 
limited use in determining the frequency 
of diagnostic error in outpatient medical 
practice. 

Population-based estimates of diagnos- 
tic error could help prioritise ongoing 
patient safety efforts. We aimed to esti- 
mate the frequency of outpatient diagnos- 
tic errors in the US adult population by 
synthesising data from three previous 
studies of clinic-based populations that 
used conceptually similar definitions of 
diagnostic error. 4-6 

METHODS 

Data sources included two previous 
studies that used electronic triggers, or 
algorithms, to detect unusual patterns of 
return visits (primary care study) or lack 
of follow-up of abnormal clinical findings 
for colorectal cancer (CRC) (colon 
cancer study), both suggestive of diagnos- 
tic errors. 4 6 A third study examined con- 
secutive cases of lung cancer in two 
institutions (lung cancer study). 5 In all 
three studies, diagnostic errors were con- 
firmed through chart review and defined 
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as missed opportunities to make a timely or correct 
diagnosis based on available evidence. The criteria for 
diagnostic errors were comparable across the three 
studies and excluded atypical presentations and appro- 
priate decisions to watch and wait. 

The triggers in the primary care study 6 were based 
on the occurrence of unexpected return visits and 
were therefore selective for misdiagnosis of acute con- 
ditions and exacerbations of chronic conditions. 
Trigger 1 was a primary care index visit followed by 
unplanned hospitalisation within 14 days; trigger 2 
was a primary care index visit followed by one or 
more unscheduled visit (s) within 14 days. Control 
visits did not meet either criterion. We considered for 
inclusion all primary care visits (n=212 165 visits and 
81 483 unique patients) in two large health systems 
between 1 October 2006 and 30 September 2007. 
For the record review, we reviewed all trigger 1 
records from one of the sites (n=220 records) and 
randomly chose the rest from the second site, yielding 
a reviewed sample of 674 records out of the total of 
1086 visits selected. Maintaining a record sampling 
ratio from each health system that was consistent with 
trigger 1, we reviewed 669 of 14 777 visits selected 
by trigger 2 and 614 of 193 810 control non-triggered 
visits. Errors in triggered and non-triggered (control) 
cases were judged to have occurred when adequate 
data to suggest the final, correct diagnosis existed at 
an earlier primary care visit, or if documented abnor- 
mal findings at the visit should have prompted add- 
itional evaluation that would have revealed the correct 
diagnosis. Determination of errors was made on retro- 
spective review by two independent raters. In extrapo- 
lating error rates to all primary care patients at our 
study sites, we assumed that rates of error in randomly 
selected samples of triggered and non-triggered cases 
were applicable to the entire cohort of triggered and 
non-triggered records in each clinic population. 

Although we presumed that the types of errors 
detected in the primary care study were likely to 
account for the majority of outpatient diagnostic 
errors, we also aimed to estimate the rate of errors 
related to initial diagnosis of less common but serious 
chronic conditions. Unlike misdiagnosis of acute con- 
ditions, these errors may not become apparent for 
weeks or months. As a proxy for such errors in out- 
patient care, we obtained frequency estimates for 
missed and delayed diagnosis of CRC and lung 
cancer, represented in the two cancer studies, 4 5 
respectively. These conditions were selected due to 
their relative frequency in several types of prior obser- 
vational studies related to diagnostic errors. 5 7-12 In 
both of our studies, diagnostic errors were charac- 
terised by lack of timely follow-up on specific 'red 
flags' or 'alarm' features 5 10 13 of cancer in patients 
who were eventually diagnosed with either cancer. 
For instance, for CRC this was operationally defined 
as lack of follow-up 60 days after documentation of 



hematochezia (bright red blood per rectum), positive 
faecal occult blood test and iron deficiency anaemia. 
For lung cancer, we defined lack of follow-up as 
failure to initiate further investigation or an appropri- 
ate action within 7 days of a documented 'red flag' 
such as an abnormal chest X-ray. None of these data 
overlapped with the primary care study. We assumed 
that failure to follow-up on these 'red flags' accounted 
for all diagnostic errors associated with these types of 
cancer. 

We extrapolated the frequency of diagnostic error 
obtained from our studies to the US adult popula- 
tion, 14 using the primary care study as a proxy to esti- 
mate rates of diagnostic error for acute conditions 
(and exacerbations of existing conditions) and the two 
cancer studies to conservatively estimate rates of 
missed diagnosis of CRC and lung cancer (as proxies 
for other serious chronic conditions). Because of the 
heterogeneity of the conditions we included, we 
assumed that they represented the spectrum of dis- 
eases at risk for outpatient diagnostic errors. 

RESULTS 

In the primary care study, samples of triggered visits 
and control visits were reviewed to determine the per- 
centages of visits that contained errors. These percen- 
tages were extrapolated to the larger triggered and 
non-triggered samples. For trigger 1, 141 errors were 
found in 674 visits reviewed, yielding an error rate of 
20.9%. Extrapolating to all 1086 trigger 1 visits 
yielded an estimate of 227.2 errors. For trigger 2, 36 
errors were found in 669 visits reviewed, yielding an 
error rate of 5.4%. Extrapolating to all 14 777 trigger 
2 visits yielded an estimate of 795.2 errors. Finally, 
for the control visits, 13 errors were found in 614 
visits reviewed, yielding an error rate of 2.1%. 
Extrapolating to all 193 810 control visits yielded an 
estimate of 4103.5 errors. Thus, we estimated that 
5126 errors would have occurred across the three 
groups. We then divided this figure by the number of 
unique primary care patients in the initial cohort 
(81 483) and arrived at an estimated error rate of 
6.29%. Because approximately 80.5% of US adults 
seek outpatient care annually, 15 the same rate when 
applied to all US adults gives an estimate of 5.06% 
(see figure 1). 

The colon cancer study estimated errors in CRC 
diagnosis in 26 out of 291 773 patients seen in out- 
patient care (0.009%). Extrapolating to the US adult 
population that seeks outpatient care, we estimated 
0.007% of all adults would have errors related to 
CRC per year. In the lung cancer study, we found 127 
errors in 587 cases of lung cancer, yielding a 21.64% 
error rate. Extrapolating to the 80.5% of the US adult 
population that seeks outpatient care and the 0.072% 
of the population that becomes newly diagnosed with 
lung cancer each year, 16 we estimate 0.013% of all 
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Primary Care Study 5 

• 6.291 % error rate (5.126 errors in 
81 .483 unique patients) in adults 
with provider visits 

• 80.5% of adults visit providers in 
outpatient care 15 



5.064% Error Rate Across All Adults 



Total US Population 2012 
n = 308,745,538 14 



Under 18 
n = 74,181.467'' 
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Colorectal Cancer Study 4 

• 0.009% error rate (26 errors in 
291 .773 unique patients) in adults 
with provider visits 

• 80.5% of adults visit providers in 
outpatient care 15 



0.007% Error Rate Across All Adults 



Lung Cancer Study 5 

• 21. 64% error rate (127 errors in 
587 lung cancer cases) in adults 
with lung cancer 

• 80.5% of adults visit providers in 
outpatient care 15 

• 0.072% of adults are newly 
diagnosed with lung cancer per 
year 16 



0.013% Error Rate Across All Adults 



Combined Estimate of Errors for Adults =5.084% 
n = -12 Million Adults per Year 



Figure 1 Schematic of synthesis of diagnostic error frequency data from three studies. 



adults would have errors related to lung cancer per 
year. 

Combining estimates from the three studies yields a 
rate of outpatient diagnostic errors of 5.08%, or 
approximately 12 million US adults every year. Based 
on our previous work, 17 we estimated that about 
one-half of errors would have the potential to lead to 
severe harm. 



DISCUSSION 

Although diagnostic errors are difficult to define and 
measure, there is mounting evidence of their import- 
ance and frequency across several types of condi- 
tions. 3 7 11 18 Applying aggregate diagnostic error 
rates from three previous studies conducted by our 
group, we estimate that approximately 12 million 
adults in the USA could experience outpatient diag- 
nostic errors each year. Our estimates are generally 
consistent with data from the general public about 
diagnostic errors. A recent telephone survey of a 
random probability sample of over 2000 US adults 
suggested that more than one in 10 respondents 
reported a 'diagnostic mishap'. 19 Similarly, a survey of 
726 paediatricians revealed that more than half (54%) 
reported making a diagnostic error at least once or 
twice per month. 20 Thus, other recent data corrobor- 
ate our finding that diagnostic errors are common. 



Previous estimations of the frequency of diagnostic 
error largely relied on expert opinion. For example, 
Elstein estimated errors occur in about 10-15% of 
diagnoses. 21 As noted by Graber, many methods to 
study diagnostic errors have limited use in calculating 
the frequency of the problem. For example, autopsy 
rates have declined, 22 the number of self-reports 
remains low and malpractice claims data are prone to 
bias and are non-representative. 11 20 23-25 On the 
other hand, population-based studies have relied on 
medical record reviews, which are generally a gold 
standard for determining diagnostic error. 26 Our elec- 
tronic trigger approach facilitated selective record 
reviews, thus making the error determination process 
more efficient than either consecutive or random 
record reviews. Our estimate, based on a large sample 
size, is likely to be the most robust one thus far to 
address the frequency of diagnostic error in routine 
outpatient practice. 

Although it is unknown how many patients will be 
harmed from diagnostic errors, our previous work 17 
suggests that about one-half of diagnostic errors have 
the potential to lead to severe harm. While this is 
only an estimate and does not imply all those affected 
will actually have harm, this risk potentially translates 
to about 6 million outpatients per year. Additionally, 
while the contribution of the two cancers to the 
overall estimation was small, we believed it was 
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important to include them to show their relative con- 
tribution. This is because delayed cancer diagnosis is 
believed to be one of the most harmful and costly 
types of diagnostic error in the outpatient setting and 
its significance has become apparent not only in mal- 
practice claims but also in retrospective studies of con- 
secutive cancer cases, surveys and studies of failures to 
follow-up abnormal test results. 5 7-12 In many of 
these studies, lung cancer and CRC are the most 
common cancers for which diagnosis is delayed. 

Our estimates are limited by our assumptions of gen- 
eralisability. To keep the definition of diagnostic error 
uniform across studies, we only used data from our 
own work. Other studies have used different methods 
and definitions of diagnostic errors and we believed 
that this would make our extrapolations weaker. 
Similar to other methods to study medical errors, our 
trigger was not sensitive enough to capture all types of 
diagnostic errors in outpatient settings. Our triggers 
did not adequately capture errors in diagnosis of 
uncommon diseases, some of which might evolve over 
a prolonged period of time in the outpatient setting. 
Our studies also might not have adequately addressed 
other types of common chronic conditions that could 
be vulnerable to diagnostic error. 27 For example, a 
recent study from the UK suggests that opportunities 
to diagnose chronic obstructive pulmonary disease at 
an earlier stage are being missed in the majority of 
cases. 28 While it is tempting to do so, we did not 
extrapolate our chronic condition proxy estimates to 
all types of cancer or other types of chronic diseases 
because it would not be possible to do this accurately. 
Lastly, chart reviews inherently would miss errors due 
to documentation-related issues and this false negative 
rate is largely unknown. All of these limitations thus 
suggest that we underestimated the frequency of diag- 
nostic errors and frequency could be well over 5%. 

In conclusion, we estimate the frequency of diag- 
nostic error to be at least 5% in US outpatient adults, 
a number that entails a substantial patient safety risk. 
This population-based estimate should provide a foun- 
dation for policymakers, healthcare organisations and 
researchers to strengthen efforts to measure and 
reduce diagnostic errors. 
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